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Abstract 


The purpose of the research is to examine the learning outcomes included in Science and Art 
Centers’ summer school support and development course of thinking education workshop 
program according to Bloom’s taxonomy. The research was designed as a qualitative study, 
utilizing the document analysis technique and benefiting from descriptive analysis. The reliability 
coefficient for the research was 0.87. The results suggested that 91 of the primary, secondary, and 
high school learning outcomes in the workshop program were related to the cognitive domain of 
Bloom’s taxonomy, and five of them were related to the affective domain. An examination of 
cognitive learning outcomes indicated that learning outcomes at the primary, secondary, and 
high school levels were at the lower levels of the taxonomy, namely remembering, understanding, 
and applying. However, there was little emphasis on learning outcomes for higher-order thinking 
skills, which include analyzing, evaluating, and creating. The same situation was observed in 
examinations carried out at the primary, secondary, and high school levels. Regarding the 
affective domain, one of the five learning outcomes was at the receiving level, while the other four 
were at the responding level. 
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1. Introduction 


Knowledge alone is insufficient in the twenty-first century education. This century 
requires the integration of knowledge with thinking for its effective utilization. Thinking is a 
lifelong ongoing process. In this lifelong process, students need to be equipped with fundamental 
knowledge and skills, along with abilities like critical thinking, analytical thinking, creative 
thinking, problem-solving, and collaborative teamwork. Thinking is the term used for the goal- 
oriented organized mental process carried out to understand the current situation (Ciiceloglu, 
1999). Thinking is one of the significant skills which fosters individuals’ language, cognitive, and 
social development, shaping their learning and guiding their future. Furthermore, thinking is the 
most crucial component of the process of acquiring knowledge, understanding, and learning, and 
lies at the heart of mental processes (Giines, 2012). Thinking forms the foundation for questioning, 
evaluating, and generating new information, requiring mental images (Arnheim, 2007). 
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Individuals concretize real-life events and objects through mental images (Yildiz, 2020). 
Therefore, it is evident that thinking guides not only individuals’ development in various areas but 
also their learning and future. In addition, people need healthy thinking in every aspect of life. 


Thinking education, on the other hand, enables the processing of knowledge, the 
culture of life, the development of culture, democratization, self-learning, understanding oneself, 
others, and the universe, and facilitates reasoning (Doganay, 2012). Furthermore, thinking 
education increases people’s quality of life. It not only enhances success in real-life situations but 
also improves academic achievement, contributes to skill development, and fosters positive 
attitudes (Giiven & Kiiriim, 2006; Tokmak, Yilmaz & Seker, 2019). Various thinking styles are 
important in education and the business sector, and individuals with good thinking skills have 
more job opportunities (Ay, 2005). The fundamental processes of thinking styles are problem- 
solving, decision-making, and critical thinking (Presseisen, 1991, as cited in Doganay, 2012). In 
the thinking process, various operations such as analysis, synthesis, comparison, generalization, 
and abstraction are commonly performed (Nickerson, 1988). One of the significant goals of 
education is to enhance students’ skills in creativity, problem-solving, critical thinking, and 
higher-order thinking. In this process, thinking operations, processes, and skills are given special 
emphasis, and efforts are made to develop thinking operations, processes, and skills through 
specially designed programs and classifications (Giines, 2012). 


One of the important classifications that play a significant role in developing thinking 
operations, processes, and skills is Bloom’s taxonomy. The taxonomy developed by Bloom is 
related to different thinking skills. Teachers believe that the higher-order thinking skills found in 
the analysis, evaluation, and creation levels of the taxonomy are necessary for all classes. Students 
who can utilize skills such as identifying, classifying, analyzing, synthesizing, and evaluating are 
considered successful (Demirkaya, 2015). In Bloom’s taxonomy, the levels of analyzing, 
evaluating, and creating are considered higher-order thinking levels (Ay, 2005; Karakas-Yildirim, 
2020; Anderson & Krathwohl, 2010; Ugurlu, 2023; Ulum, 2017). However, levels of remembering, 
understanding, and applying are also necessary for higher-order thinking. This is because lower- 
order thinking skills transform into increasingly higher-order skills (Biger, 2019). In Bloom’s 
taxonomy, each level is related to one another and not entirely independent. Cognitive domain 
skills are arranged from simple to complex. According to Unalmis (2023), thinking skills are also 
interconnected, just like in Bloom’s taxonomy. In this context, it could be stated that there should 
be learning outcomes related to each level of Bloom’s taxonomy in the thinking workshop 
program. However, it could be argued that there should be more learning outcomes focused on 
developing higher-order thinking skills in particular. 


In recent years, thinking skills education has become a significant problem in 
countries, and there have been significant challenges in providing thinking skills education, and 
the practices in this field have been inadequate (Baysal, Carikci & Yasar, 2016; Bicer, 2019; Giines, 
2012). Thinking is innate and can be developed through appropriate education (Costa, 2016). An 
educational program that places particular emphasis on thinking can be highly effective in 
developing thinking skills (Ipsiroglu, 2015). In this educational program, it is crucial to determine 
the learning outcomes appropriately. The Science and Art Centers (SAC) Summer School Support 
and Development Course of Thinking Education Workshop Program is an important practice in 
terms of thinking skills education. Therefore, it is considered important to examine the learning 
outcomes in this workshop program according to Bloom’s taxonomy. 


There are two important international practices related to students’ higher-order 
thinking skills. One of these practices is the Program for International Student Assessment (PISA). 
PISA measures students’ knowledge and skills in mathematics, science, and reading, assessing 
their ability to apply these skills to overcome real-life challenges. Another practice is the Trends 
in International Mathematics and Science Study (TIMSS). TIMSS aims to assess students’ 
knowledge and skills in mathematics and science and contribute to their development in these 
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areas (IEA, 2023). These two practices are based on higher-order thinking skills (IEA, 2023; 
OECD, 2023; Yilmaz, 2019). One of the similar practices is the Academic Skills Monitoring and 
Evaluation (ABIDE) project implemented in Turkey. The ABIDE project aims to determine the 
extent to which students possess higher-order thinking skills through various types of questions. 
Accordingly, the ABIDE project aims to assess the higher-order mental skills of 8 grade students 
in Turkish language, mathematics, science, and social studies (MEB, 2023). Examining the 
learning outcomes included in the Science and Art Centers’ summer school support and 
development course program and facilitating the acquisition of critical thinking skills will also 
increase success in national and international exams. 


Workshops for SAC summer school programs were first introduced in the summer of 
2022. The workshop programs are implemented separately for primary school, secondary school, 
and high school levels, and all students from grades 2-12 can participate in the workshops. One of 
these programs is the Thinking Education Workshop. The objectives of the Thinking Education 
Workshop are to enable students to utilize 21t century skills and higher-order thinking skills, 
establish their own thinking systems, establish a connection between language and thinking, 
analyze events and phenomena, express themselves effectively, respect different perspectives, 
think in a multifaceted manner, and support collaborative problem-solving (MEB, 2022). The 
general objective of the workshop is to encourage students to think about thinking, establish their 
own thinking systems, recognize that thinking is a skill that can be developed, consider national 
and universal values while thinking, respect different perspectives, become aware of what they 
know and do not know, realize the importance of meeting the thinking demands of the 21* century, 
and enable them to express themselves effectively through effective communication (Ersoy, Kefeli, 
Parmaksiz, Karaman & Duran, 2022). Therefore, the thinking education workshop has an 
important function. It is also considered essential to examine the learning outcomes included in 
this workshop. In the context of thinking education, Sonmez (2016) conducted research on the 
impact of the 6'-grade elective Critical Thinking Education course on primary school students’ 
critical and creative thinking skills. There were also many research studies in the literature 
examining the learning outcomes included in the curriculum in line with Bloom's Taxonomy 
(Biiyiikalan Filiz & Yildirim, 2019; Cerci, 2018; Karagol, 2020; Lee, Kim & Yoon, 2015). In 
addition, numerous studies have examined the questions asked in lessons and exams (Eke, 2015; 
Cintas-Yildiz, 2015; Demir, 2015; Gokulu, 2015; Giilleryiiz, 2016; Kana & Giiney, 2020; Kamk- 
Uysal, 2022; Sanh & Pinar, 2017; Yildiz, 2015; Zhang, Wong, Giacaman & Luxton-Reilly, 2021), 
and activities according to Bloom’s Taxonomy (Bayrak-Ozmutlu & Kamk-Uysal, 2021; Crompton, 
Burke & Lin, 2018; Durukan & Demir, 2017; Eroglu, 2013; Giiltekin, 2019; Karakas-Yildirim, 
2020; Pujawan, Rediani, Antara, Putri & Bayu, 2022; Ulum, 2017). However, no research could 
be found regarding the examination of the learning outcomes given in the Thinking Education 
Workshop within the scope of SAC summer school programs. 


As the Thinking Education Workshop delivered within the scope of SAC summer 
school programs is the first in its kind and no study has examined the workshop learning outcomes 
in the literature, investigating the distribution of primary school, secondary school, and high 
school learning outcomes included in the summer school thinking education workshop program 
according to Bloom’s taxonomy levels is considered a worthwhile subject for research. 


This research examines the learning outcomes in the “Thinking Education Workshop” 
program, which is part of the SAC Summer School Support and Development Course Program 
prepared by the Ministry of National Education (MoNE). In this context, the research aims to 
examine the learning outcomes of the summer school Thinking Education Workshop program 
according to Bloom’s taxonomy. Therefore, answers were sought to the following questions: 


1. What is the distribution of primary school learning outcomes in the 
Summer School Thinking Education Workshop Program according to 
Bloom’s taxonomy process levels? 
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2. What is the distribution of secondary school learning outcomes in the 
Summer School Thinking Education Workshop Program according to 
Bloom’s taxonomy process levels? 


3. What is the distribution of high school learning outcomes in the Summer 
School Thinking Education Workshop Program according to Bloom’s 
taxonomy process levels? 


4. What is the distribution of all learning outcomes in the Summer School 
Thinking Education Workshop Program according to Bloom’s taxonomy 
process levels? 


2. Method 


This study employed a qualitative research design using document analyses. 
Document analysis is the examination of written documents concerning a specific phenomenon 
or event intended to be explored (Yildirim & Simsek, 2016). In qualitative studies, documents are 
an important source of data and can include both private and official documents (Creswell, 2017). 
The document analysis method, which focuses on how the researched subject is reflected in 
documents, is commonly used in educational research. Document materials can include public 
records, textbooks, letters, films, cassettes, diaries, themes, reports, or other documents (Ary, 
Jacobs & Sorensen, 2010). In this study, the data source document considered is the 2022 
Thinking Education Workshop Summer School Program prepared by the MoNE for SACs under 
the Summer School Support and Development Course Program. 


2.1 Data analysis 


In this study, descriptive analysis was used to examine the learning outcomes of focus. 
In this analysis method, the data obtained from the document are examined and interpreted 
according to pre-determined categories, themes, or dimensions. This examination process is 
carried out in four steps. In step 1, a framework is established to determine under which themes 
or concepts the data will be organized. In step 2, within the specified thematic framework, the 
document is read, and the data are organized and processed. In step 3, the findings organized 
according to this framework are supported with quotations. In the final step, the findings are 
explained, related to each other, and interpreted to make them more meaningful (Yildirim & 
Simsek, 2016). 


In this respect, first, a framework was established for analyzing the data based on the 
cognitive process dimensions of Bloom’s taxonomy. Then, the following steps were followed. In 
the first stage of document analysis, the learning outcomes included in the SAC Summer School 
Thinking Education Workshop Program, one of the 2022 SAC Summer School Workshop 
Programs published by the General Directorate of Special Education and Guidance Services, were 
examined according to primary, secondary, and high school levels, and their frequencies were 
extracted. As a result of this examination, a total of 96 learning outcomes were examined in this 
research, with 34 learning outcomes at the primary school level, 30 at the secondary school level, 
and 32 at the high school level. Then, action statements corresponding to the cognitive process 
levels of the revised Bloom’s taxonomy were determined, and the learning outcomes were 
examined according to the revised Bloom’s taxonomy. Ninety-six learning outcomes were 
separately coded by the researcher and an expert with a Ph.D. degree in educational sciences. The 
reliability between the two coders was determined using Miles and Huberman’s (2015) reliability 
formula (Reliability = Consensus/Consensus + Disagreement). According to this formula, the 
reliability concerning the learning outcomes in the cognitive process dimension was 0.87. When 
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examining the learning outcomes, the cognitive process represented by the action word at the end 
of each learning outcome statement was determined. For example, in the learning outcome, “2.19. 
Realizes that there are various problems in daily life”, the action word is “realizes.” This statement 
emphasizes a cognitive process at the level of remembering. The learning outcome “1.12. Conducts 
practices related to reasoning” is examined as the application level in terms of the cognitive 
process dimension. 


3. Findings 


This study examined the distribution of primary, secondary, and high school learning 
outcomes included in the SAC Summer School Thinking Education Workshop Program according 
to the cognitive dimensions of Bloom's taxonomy. 


3.1 Findings regarding the cognitive process dimensions of primary school 
learning outcomes included in the thinking education workshop program 


In the analysis conducted regarding the first research question, it was determined that 
the 33 primary school learning outcomes included in the SAC Summer School Thinking Education 
Workshop Program were related to the cognitive domain, and one was related to the affective 
domain at the receiving level. Findings regarding the cognitive dimensions of learning outcomes 
are presented in Figure 1. 


Distribution of Primary School Learning Outcomes 
by Cognitive Process Dimensions in Percentage 


Percentage 


18.2 


9.1 

6.1 6.1 7 
—_ 
‘ lh a 
Remembering Understanding Applying Analyzing Evaluating Creating 


Cognitive Process Dimension 


Figure 1. Findings regarding the cognitive process 
dimensions of primary school learning outcomes 


As examined in Figure 1, out of the 33 primary school learning outcomes related to the 
cognitive domain, 19 (57.5%) were at the remembering level, 6 (18.2%) at the understanding level, 
2 (6.1%) at the applying level, 2 (6.1%) at the analyzing level, 3 (9.1%) at the evaluating level, and 
1 (3%) at the creating level. The fact that 81.8% of the primary school learning outcomes were at 
the remembering, understanding, and applying levels indicates that the thinking education 
program provides students with lower-order skills according to Bloom’s taxonomy. The program 
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had little emphasis on higher-order thinking skills of analyzing, evaluating, and creating levels 
(18.2%). 


3.2 Findings regarding the cognitive process dimensions of secondary 
school learning outcomes included in the thinking education workshop 
program 


In the analysis conducted regarding the second research question, it was determined 
that 29 of the secondary school learning outcomes included in the SAC Summer School Thinking 
Education Workshop Program were related to the cognitive domain, and one was related to the 
affective domain at the receiving level. Findings regarding the cognitive dimensions of learning 
outcomes are presented in Figure 2. 


Distribution of Secondary School Learning Outcomes 
by Cognitive Process Dimensions in Percentage 


Percentage 


48.3 
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Cognitive Process Dimension 


Figure 2. Findings regarding the cognitive process 
dimensions of secondary school learning outcomes 


As seen in Figure 2, out of the 29 secondary school learning outcomes related to the 
cognitive domain, 14 (48.3%) were at the remembering level, 5 (17.2%) at the understanding level, 
5 (17.2%) at the applying level, 2 (6.9%) at the analyzing level, 1 (3.5%) at the evaluating level, and 
2 (6.9%) at the creating level. The fact that 87.2% of the secondary school learning outcomes were 
at the remembering, understanding, and applying levels indicates that the thinking education 
program provides students with lower-order skills according to Bloom’s taxonomy. The program 
had little emphasis on higher-order thinking skills of analyzing, evaluating, and creating levels 
(17.3%). 


3.3 Findings regarding the cognitive process dimensions of high school 
learning outcomes included in the thinking education workshop program 


In the analysis conducted regarding the third research question, it was determined 
that 29 of the high school learning outcomes included in the SAC Summer School Thinking 
Education Workshop Program were related to the cognitive domain, and three were related to the 
affective domain. Findings regarding the cognitive dimensions of learning outcomes are presented 
in Figure 3. 
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Distribution of High School Learning Outcomes 
by Cognitive Process Dimensions in Percentage 
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Figure 3. Findings regarding the cognitive process dimensions of high school learning outcomes 


As illustrated in Figure 3, out of the 29 high school learning outcomes related to the 
cognitive domain, 29 (65.5%) were at the remembering level, 4 (13.2%) at the understanding level, 
1 (3.4%) at the applying level, 2 (6.9%) at the evaluating level, and 3 (10.4%) at the creating level. 
The fact that 82.7% of the high school learning outcomes were at the remembering, understanding, 
and applying levels indicates that the thinking education program provides students with lower- 
order skills according to Bloom’s taxonomy. The program included no learning outcome 
concerning the higher-order thinking skill of the analysis level, while it had included very few 
learning outcomes at the analyzing, evaluating, and creating levels (17.3%). 


3.4 Findings regarding the cognitive process dimensions of all learning 
outcomes included in the thinking education workshop program 


In the analysis conducted regarding the fourth research question, it was determined 
that out of the 96 learning outcomes included in the SAC Summer School Thinking Education 
Workshop Program, 91 were related to the cognitive domain and five were related to the affective 
domain. Findings regarding the levels that these learning outcomes were related to are presented 
in Table 1. 


Table 1. Findings regarding the cognitive process 
dimensions of the workshop program learning outcomes 


Learning Domain Levels a of Learning % 
utcomes 

Remembering 52 54.16 
Understanding 15 15.62 

aos : Applying 8 8.33 
Cognitive domain Ansiyainie 4 4.16 
Evaluating 6 6.25 

Synthesizing 6 6.25 

i : Receiving 1 1.04 
Affective domain Responding r 4.16 
Total 96 100 
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According to Table 1, 52 (54.16%) learning outcomes in the cognitive domain were at 
the remembering level, 15 (15.62%) at the understanding level, 8 (8.33%) at the applying level, 4 
(4.16%) at the analyzing level, 6 (6.25%) at the evaluating level, and 6 (6.25%) at the synthesizing 
level. In total, there were 75 (78.11%) learning outcomes at remembering, understanding, and 
applying levels. There were 16 (16.66%) learning outcomes related to the higher-order skills of 
analyzing, evaluating, and creating. Of five learning outcomes in the affective domain, one was at 
the receiving level and the other four were at the responding level, making up 5.2% of all learning 
outcomes. 


4. Discussion, conclusion, and recommendations 


The study concluded that 91 primary, secondary, and high school learning outcomes 
in the SAC Summer School Thinking Education Workshop Program were related to the cognitive 
domain of Bloom’s taxonomy and five were related to the affective domain. Of the five affective 
learning outcomes, one was at the receiving level and four were at the responding level. According 
to Dewey (1993), affective characteristics such as personal interest, sincerity, open-mindedness, 
and taking on responsibility are influential in the thinking process. According to Piaget (2004), 
affective characteristics such as love, interest, values, and impressions of harmony are important 
in the thinking process. According to Costa (2016), empathy and emotions are also influential 
factors in our thinking. The act of thinking cannot be independent of emotions, and thus equal 
attention should be given both to the affective and cognitive domains (Ornestein & Hunkins, 
2016). However, only five learning outcomes were related to the affective domain, indicating that 
affective learning outcomes are insufficient in the workshop program. Therefore, the number of 
affective learning outcomes should be increased in new programs designed for thinking education. 


The research found that most of the primary, secondary, and high school learning 
outcomes were at the lower levels of Bloom’s taxonomy, namely remembering, understanding, 
and applying. There was very little emphasis on learning outcomes related to higher-order 
thinking skills like analyzing, evaluating, and creating. However, in the State of Georgia’s list of 
thinking skills, there are four learning outcomes at the level of remembering, five at the level of 
understanding, six at the level of applying, eight at the level of analyzing, six at the level of 
evaluating, and five at the level of creating. This list is presented as an example of a comprehensive 
thinking education program by Oliva and Gordon (2018). According to Bloom (1956), complex 
behaviors encompass simpler behaviors. As the levels progress in the taxonomy, there will be more 
complex behaviors and thinking processes. Therefore, higher-order thinking skills encompass the 
learning outcomes at the levels of analyzing, evaluating, and creating in Bloom’s taxonomy 
(Akkus-Cakir & Senemoglu, 2016; Akyol & Kilic, 2021; Ay, 2005; Anderson & Krathwohl 2010; 
Berger, 2018; Karakas-Yildirim, 2020; Stayanchi, 2017; Ugurlu, 2023; Ulum, 2017). This situation 
indicates the learning outcomes in the thinking education workshop are insufficient for developing 
higher-order thinking skills. Therefore, the new programs to be prepared for thinking education 
should include more learning outcomes related to higher-order thinking skills such as analyzing, 
evaluating, and creating. 


Since the SAC Summer School Support and Development Course of Thinking 
Education Workshop Program was first published in 2022, no research was found in the literature 
regarding the learning outcomes of the workshop program. However, similar studies examining 
the learning outcomes of different subjects’ curricula according to Bloom’s taxonomy are available 
in the literature. These studies examining the curricula of Turkish, social studies, science, history, 
geography, physics, chemistry, and biology subjects indicate that the learning outcomes consist of 
the levels of remembering, understanding, and applying according to Bloom's taxonomy (Aktan, 
2020; Avsar & Mete, 2018; Biiken & Artvinli, 2021; Biiyiikalan Filiz & Baysal, 2019; Biiyiikalan 
Filiz & Yildirim, 2019; Celik, Kul & Calik-Uzun, 2018; Cerci, 2018; Dogan & Burak, 2018; Durukan 
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& Demir, 2017; Eroglu, 2013; Gildtiren & Cangiiven, 2020; Giltekin & Burak, 2019; Karagol, 
2020; Ilhan & Gilersoy, 2019; Karakas-Yildirim, 2020; Onlen, Tatan & Ibret, 2020; Ozdemir, 
Altiok & Baki, 2015; Sozcii & Aydin6zii, 2019; Unveren-Kapanadzade, 2019; Zorluoglu, Sahintiirk, 
& Bagriyamik, 2017). These findings are similar to the results of this research. Contrary to these 
research findings, Gezer, Sahin, Siinktir and Meral (2014) concluded that the learning outcomes 
of the Turkish Republic History of Revolution and Kemalism Course curriculum mostly consisted 
of higher cognitive levels. The researchers suggest that thinking education programs should be 
prepared by incorporating higher-level learning outcomes, and the impact of these programs 
should be investigated. 


The study showed that approximately 82% of the primary school learning outcomes 
consisted of lower-order learning outcomes, while 18% were higher-order learning outcomes. On 
the other hand, around 83% of secondary and high school learning outcomes consisted of lower- 
order learning outcomes, while 17% were higher-order learning outcomes. These results show that 
the learning outcomes at the primary, secondary, and high school levels are similar in terms of 
lower and higher-order learning outcomes. Indeed, as one progresses from lower to higher grades, 
the level of learning outcomes should increase (Anderson & Krathwohl, 2010). This indicates that 
student characteristics and the principle of progressivity were not adequately taken into account 
in formulating the learning outcomes included in the workshop program. However, in programs 
designed for thinking education, student characteristics should be taken into consideration, and 
the learning outcomes should be included progressively. 


The present-day world necessitates individuals to possess some thinking skills. 
Individuals with higher-order thinking skills are advantageous over those who can only directly 
acquire the transmitted information. Especially in developed societies, the primary purpose of 
education is to raise individuals who are sensitive to problems, capable of problem-solving, and 
possess higher-order thinking skills (Bapoglu, 2010). Therefore, educational programs, especially 
the thinking education workshop program, should aim to raise individuals who can access 
information, think critically, and produce. However, it could be stated that the current state of the 
Thinking Education Workshop program is not sufficiently serving the purpose of raising the 
expected individuals. 
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